feat(ai-plugin): implement AI model auto-detection and enhance plugin loading reliability #837

KariHall619 · 2025-09-28T15:28:35Z

What type of PR is this?
implement AI model auto-detection and enhance plugin loading reliability

What this PR does / why we need it:

This PR introduces comprehensive AI plugin integration capabilities with intelligent model detection and robust plugin loading mechanisms. The changes address critical reliability issues in plugin initialization and provide a foundation for dynamic AI model management.

Key Features:

AI Model Auto-Detection System (pkg/testing/model_detector.go)
- Implements automatic detection of available AI models across multiple providers (OpenAI, Claude, local models)
- Eliminates the need for manual model configuration
- Provides comprehensive error handling and logging
Enhanced Plugin Loading Reliability (console/atest-ui/src/views/Extension.vue)
- Implements exponential backoff retry mechanism (up to 10 attempts with incremental delays: 50ms, 100ms, 150ms...)
- Proper loading state management to prevent UI freezing
- Comprehensive error handling with detailed logging
- Optimized CSS/JS loading sequence
AI Plugin Communication Infrastructure (pkg/testing/remote/grpc_store.go)
- Extends gRPC store to support AI operations (ai.generate, ai.capabilities)
- Implements parameter encoding and response conversion for AI queries
- Standardizes AI response format for consistent integration
Configuration Optimization (cmd/testdata/stores.yaml)
- Updates AI plugin to use Unix socket communication (unix:///tmp/atest-ext-ai.sock)
- Enables auto-detection with improved model description
- Removes hardcoded model defaults to support dynamic detection
Development Environment Improvements
- Fixed extension menu parameter passing in frontend routing
- Updated Vite configuration to avoid port conflicts (5173 → 5174)
- Complete unit test coverage for model detection functionality

Why we need it:

Reliability: The original plugin loading was fragile and failed when plugins needed initialization time
User Experience: Auto-detection eliminates manual configuration and reduces setup complexity
Scalability: Provides a robust foundation for supporting multiple AI providers and models
Maintainability: Comprehensive error handling and logging improve debugging and monitoring

Testing:

Added 91 lines of unit tests for model detection functionality
Covers various detection scenarios and edge cases
Ensures stability and reliability of the AI integration system

This PR transforms the AI plugin system from a basic proof-of-concept into a production-ready, intelligent, and reliable integration platform.

KariHall619 · 2025-09-28T16:24:39Z

I am thinking about whether model-detectors should be incorporated into the ai plugin...

console/atest-ui/src/views/Extension.vue

console/atest-ui/src/App.vue

pkg/testing/model_detector.go

Adds support for AI-powered features via a new extension mechanism. This includes dynamically loading the AI extension's CSS and JS, and retrying the plugin mounting process with exponential backoff to ensure proper initialization. Also, provides a basic GRPC querying functionality which could call AI methods, and converts the AI response to standard data format. The AI plugin can be built from source or downloaded from a binary URL.

Restore two essential bug fixes that were incorrectly removed: 1. vite.config.ts fixes: - Fix test-id removal logic: only remove in production, preserve for E2E tests - Improve build performance: replace single chunk with optimized chunk splitting - Separate vue, element-plus, and vendor chunks for better caching 2. App.vue fix: - Fix Extension component prop: use menu.index instead of menu.name - Ensures AI plugin can be correctly identified and loaded These fixes are critical for AI plugin functionality and should not be reverted.

Consolidates the vite build configuration to output a single chunk, simplifying the config and potentially improving build times in some scenarios. The previous manual chunk configuration is removed.

Updates the AI model name description to indicate that the model is automatically detected from available models. Sets the default value for the model to an empty string, reflecting the auto-detection behavior.

…AI parameters. Adjust the stores.yaml file structure and add more configuration parameters such as provider and endpoint for AI plug-ins.

The AI plugin build process is removed from the makefile. The AI plugin is assumed to be pre-built or handled by a separate process, simplifying the build and execution flow.

console/atest-ui/src/views/Extension.vue

pkg/testing/remote/grpc_store.go

LinuxSuRen · 2025-09-30T07:24:49Z

pkg/testing/remote/grpc_store.go

+	params := map[string]string{
+		"model":  query["model"],
+		"prompt": query["prompt"],
+		"config": query["config"],
+	}


Do you think it's necessary? It looks like you just convert the map data into a JSON string.

Good catch! Added comments in 6171a02 to clarify the intent.
The function intentionally filters out the "method" field (used for routing)
before encoding - it's not just a simple map-to-JSON conversion.

(Or we can use Go structs instead of maps to provide better type safety?)

我的意思是，你把一个 map 中的部分 key-value 拿出来组成一个新的 map 是不是没有必要呢，直接用原始的 map 会有什么问题吗

我这边补充一下背景：query 里包含了一个用于内部路由的 method 字段，下游真实需要的只是生成模型调用的业务参数。如果直接把原始 map 拿去编码，下游就会看到这个路由字段，既没用又容易引发误解（他们可能以为要去处理这个字段），日志和数据记录也会因此变得杂乱。所以我才把需要的 key-value 挑出来，确保传出去的是干净的业务负载。

如果您觉得重新构建一个 map 不够好，有几个改进方向，可以选一：

复制原始 map，再显式删除 method（例如 params := maps.Clone(query); delete(params, "method")），代码更简洁；

或者直接换成一个结构体，method 用 json:"-" 标注忽略，这样类型安全也更好, 会稍微麻烦一点.

或者直接把method字段传下去.

您觉得那个好点? 还是说保持现在的逻辑呢

pkg/testing/remote/grpc_store.go

- Remove console.log/error from Extension.vue for production readiness - Add comments to encodeAIGenerateParams explaining field filtering logic Resolves review feedback from yuluo-yx and LinuxSuRen

sonarqubecloud · 2025-10-12T11:25:18Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

LinuxSuRen reviewed Sep 29, 2025

View reviewed changes

LinuxSuRen added the ospp 开源之夏 https://summer-ospp.ac.cn/ label Sep 29, 2025

KariHall619 added 5 commits September 29, 2025 14:45

Simplifies chunking config

05cd94d

Consolidates the vite build configuration to output a single chunk, simplifying the config and potentially improving build times in some scenarios. The previous manual chunk configuration is removed.

fix: Clarifies AI model name description

85a21a6

Updates the AI model name description to indicate that the model is automatically detected from available models. Sets the default value for the model to an empty string, reflecting the auto-detection behavior.

refactor(testdata): Update stores.yaml configuration to support more …

81359b5

…AI parameters. Adjust the stores.yaml file structure and add more configuration parameters such as provider and endpoint for AI plug-ins.

KariHall619 force-pushed the fix branch from 076cef5 to 81359b5 Compare September 29, 2025 06:50

KariHall619 requested a review from yuluo-yx as a code owner September 29, 2025 06:50

Removes AI plugin build process

95a5496

The AI plugin build process is removed from the makefile. The AI plugin is assumed to be pre-built or handled by a separate process, simplifying the build and execution flow.

KariHall619 mentioned this pull request Sep 29, 2025

Complete AI Extension Plugin Implementation for API Testing Framework LinuxSuRen/atest-ext-ai#21

Open

yuluo-yx reviewed Sep 29, 2025

View reviewed changes

console/atest-ui/src/views/Extension.vue Show resolved Hide resolved

yuluo-yx reviewed Sep 29, 2025

View reviewed changes

console/atest-ui/src/views/Extension.vue Outdated Show resolved Hide resolved

yuluo-yx reviewed Sep 29, 2025

View reviewed changes

pkg/testing/remote/grpc_store.go Show resolved Hide resolved

LinuxSuRen reviewed Sep 30, 2025

View reviewed changes

KariHall619 added 2 commits October 10, 2025 13:28

refactor: address code review feedback

6171a02

- Remove console.log/error from Extension.vue for production readiness - Add comments to encodeAIGenerateParams explaining field filtering logic Resolves review feedback from yuluo-yx and LinuxSuRen

refactor: align ui helpers with code quality guidance

be856f5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ai-plugin): implement AI model auto-detection and enhance plugin loading reliability #837

feat(ai-plugin): implement AI model auto-detection and enhance plugin loading reliability #837

KariHall619 commented Sep 28, 2025

Uh oh!

KariHall619 commented Sep 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LinuxSuRen Sep 30, 2025

Uh oh!

KariHall619 Oct 10, 2025

Uh oh!

LinuxSuRen Oct 15, 2025

Uh oh!

KariHall619 Oct 16, 2025

Uh oh!

Uh oh!

sonarqubecloud bot commented Oct 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat(ai-plugin): implement AI model auto-detection and enhance plugin loading reliability #837

Are you sure you want to change the base?

feat(ai-plugin): implement AI model auto-detection and enhance plugin loading reliability #837

Conversation

KariHall619 commented Sep 28, 2025

Uh oh!

KariHall619 commented Sep 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LinuxSuRen Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

KariHall619 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

LinuxSuRen Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

KariHall619 Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sonarqubecloud bot commented Oct 12, 2025

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants